A competitive ensemble pruning approach based on cross-validation technique

Ensemble pruning is crucial for the considerations of both efficiency and predictive accuracy of an ensemble system. This paper proposes a new Competitive measure for Ensemble Pruning based on Cross-Validation technique (CEPCV). Firstly, the data to be learnt by neural computing models are mostly drifting with time and environment, while the proposed CEPCV method can realize on-line ensemble pruning, taking full advantage of potentially valuable information. Secondly, CEPCV algorithm naturally inherits the predominance of the technique of cross-validation, which implies that those networks regarded as winners and selected into the pruned ensemble have the “strongest” generalization capability. Thirdly, CEPCV algorithm is essentially based on the strategy of “divide and rule, collect the wisdom”, which might alleviate the local minima problem of many conventional ensemble pruning approaches at the cost of a little greater computational cost, which is acceptable to most applications of ensemble learning. Experimental results on six benchmark classification tasks demonstrate that the proposed CEPCV algorithm is a well-tried ensemble pruning algorithm, which significantly improves the generalization and classification capability of the original ensemble.